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DETAILED ACTION 

Oath/Declaration 

1 . The oath or declaration is defective. A new oath or declaration in compliance with 37 
CFR 1.67(a) identifying this application by application number and filing date is required. See 
MPEP §§ 602.01 and 602.02. 

The oath or declaration is defective because: 

It does not state that the person making the oath or declaration acknowledges the duty 
to disclose to the Office all information known to the person to be material to patentability 
as defined in 37 CFR 1.56. 

Priority 

2. Receipt is acknowledged of papers submitted under 35 U.S.C. 1 19(a)-(d), on 8/1 1/2004, 
and which have been placed of record in the file. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21 (2) 
of such treaty in the English language. 

4. Claims 1, 5, 7-10, 14, & 15 are rejected under 35 U.S.C. 102(e) as being anticipated by 
August et al. (US 7,149,690 B2), hereinafter known as August. 
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5. August discloses an interactive computer-assisted language learning method 
implemented using a language learning system (5:17-35), the method comprising: the language 
learning system pronouncing a first sentence according to a first acoustic signal stored inside 
the language learning system (the invention converts input text data to audible speech data, 
3:52-54); a learner producing a first pronunciation by imitating the pronunciation produced by 
the language learning system so that a first pronunciation signal is recorded by the language 
learning system (the invention prompting the user to replicate the audible speech, recognizing 
utterances generated by the user in response to the prompt, 3:58-60; the student will read the 
sentences and they will be recorded, 3:63-66); the language learning system comparing the first 
acoustic signal with the first pronunciation signal to record and output a first compared result 
(Feedback is then provided to the student based on the comparison and a confidence measure, 
8:23-48); the learner producing a second pronunciation according to the first compared result so 
that the a second pronunciation signal is recorded by the language learning system (the system 
will indicate to the student that there is a problem and will help the student to decide if he or she 
should repeat the task, 11:1-15); the language learning system comparing the first acoustic 
signal with the second pronunciation signal to record and output a second compared result (the 
student could return to the original word, sub-word, or group being drilled, and continue to 
practice, 12:33-43); and the language learning system analyzing and summarizing the first and 
the second compared result to provide a comment to the learner as a reference (once the 
student has been practicing for some period of time, he or she can again request feedback. A 
summary can be created to provide feedback to the student, including overall scores, 1 1:16-26; 
August also teaches comments as reinforcement help, such as messages, 11:62-12:8) [Claim 

1]. 
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6. August discloses wherein the method further comprises the step of: the language 
learning system outputting the pronunciation of a plurality of second sentences according to 
second acoustic signals stored inside the language learning system, wherein all the second 
sentences belong to a group having a pronunciation that can easily confuse with the 
pronunciation of the first sentence (An analysis of known errors of pronunciation will be used to 
assist the student. Word sound drills are used to locate and display/play vocabulary words from 
either language, to illustrate sound-alike comparison to the student, 12:16-33) [Claim 5]. 

■ 

7. August discloses wherein the comment is presented to the learner by displaying text on 
a screen or vocalizing the commenting words (the student sees the {reinforcement help} 
message in the window, and also hears the teacher speak the message, 11:62-12:15) [Claim 7]. 

8. August discloses wherein the method further comprises the language learning system 
designing a vocal training program according to the recorded first and second compared result 
(Another practice feature associated with the workspace is an option to list practice sub-words, 
words, or groups in a window, and permit practice of sounds relating to the specific problem 
encountered by the student, 12:33-43; inherently, August teaches designing a training program 
based on analysis of a student's specific problems) [Claim 8]. 

9. August discloses a language learning system, comprising: a voice recognition engine 
(ASR and TTS modules) for recognizing the pronunciation of a plurality of first sentences 

• 4 

produced by a learner and outputting a first pronunciation signal corresponding to each 
pronunciation of the first sentence (the method is initiated with the input of text and subsequent 
conversion of the input text to audible speech data. Audible speech is generated and output 
based on the audible speech data. These models are generated primarily by the ASR module 
and associated elements. However, in certain circumstances, the TTS module may also be 
used to generate the acoustic models. A student is then prompted to replicate the audible 
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speech with spoken words, or utterances. The system then recognizes the utterances of the 
student and compares these utterances to the audible speech data primarily through the 
module, 8:23-48); a database (pronunciation module) for holding a plurality of first acoustic 
signals with each first acoustic signal corresponding to a pronunciation of the first sentence (The 
pronunciation module includes databases containing records for words, word subgroups, 
vocabulary words used to teach typical sounds in a language, examples from parts of speech 
used to teach contextual pronunciation of words and tables of punctuation. The sample words 
are selected in creating the pronunciation databases based on grammatical and linguistic rules 
for the language, 7:36-42); an analysis/processing unit (VIV portion, 6:54-7:6) connected to the 
voice recognition engine and the database for comparing the first pronunciation signals output 
from the voice recognition engine and the first acoustic signals retrieved from the database to 
produce and output a compared result (This preferred form of the ASR module having the 
verbal information verification (VIV) portion compares the output of phonemes processed by the 
TTS engine and voice, its own acoustic model or any derived acoustic model, or utterances, 
spoken by the student. This comparison provides the basis of the feedback to the student, 6:54- 
7:6; also, the feedback preferably reflects the precision at which the user replicates the audible 
speech in the selected language, 8:23-48), outputting a comment to the learner according to the 
compared result (reinforcement help, such as messages, 11:62-12:8), and designing a special 
program for training the pronunciation of the learner (individual word parts or phoneme matches 
are analyzed to indicate where precisely the student may be having difficulty in pronunciation, 
6:54-7:6; also, feedback is then provided to the student based on the comparison and a . 
confidence measure which is correlated to customized scoring tables and used as a calibration 
point, 8:23-48); and a voice-synthesizing unit (the system includes a text-to-speech module, or 
TTS module, 5:40-42) connected to the analysis/processing unit (The VIV portion analyzes the 
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* 

similarity with which a speaker matches the file created by the TTS module, 8:23-48) for 
converting the first sentence output from the database into a vocal sound (The TTS module has 
associated therewith a rules module for facilitating the conversion of the text to audible speech. 
More specifically, the rules module has stored therein code that allows multilevel analysis of the 
words for which conversion to audible speech is sought, 5:64-6:20) [Claim 9]. 

10. August discloses wherein the system further comprises a learning interface connected to 

* 

the voice recognition engine and the voice- synthesizing unit for receiving a command from the 
learner and sending the command to the analysis/ processing unit so that the 
analysis/processing unit selects part of the acoustic signals from the database according to the 
command to produce a learning program and present the program on the learning interface 
[Claim 10], and wherein the command comprises a voice command or a textual command 
[Claim 11] (The student can use a tool to scroll through the {word and punctuation practice} list. 
The student can highlight and select a word with a voice command or mouse. When the 
phoneme, word, or group is highlighted, the teacher's voice is heard pronouncing the word, 
1 1:27-61; selecting a word with the mouse is understood to be a textual command) [Claims 10 & 
11]. 

1 1 . August discloses wherein the database further holds a plurality of second acoustic 
signals each corresponding to the pronunciation of a second sentence such that the 
pronunciation of each second sentence is similar to the pronunciation of the first sentence (Text 
or audible help for this feature may be presented in the native language or the target language, 
a combination, or both. The pronunciation files may include a table of sub-words and 
corresponding sub-words in another language, to illustrate sound-alike comparison to the 
student, 12:22-33; August discloses where this applies to full sentences in 5:64-6:9) [Claim 14]. 



4 
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12. August discloses wherein the first sentence is a single word, a phrase or a full sentence 
(The rules module sequentially analyzes a selected word, analyzes the word in the context of 
the sentence, and then analyzes the sentence format, 5:64-6:9). [Claim 15]. 



Claim Rejections -35 USC §103 

13. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

• 

14. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 
(1966), that are applied for establishing a background for determining obviousness under 35 
U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating obviousness 
or nonobviousness. 



15. Claims 2-4 & 12 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
August, in view of Jochumson (US 6,453,290 B1), hereinafter known as Jochumson. 

16. August teaches all the features as demonstrated above in the rejection of claims 1 & 9- 
1 1 , including inputting a voice command [Claim 3] and a textual command [Claim 4] (The 
student can use a tool to scroll through the list. The student can highlight and select a word with 
a voice command or mouse, 1 1 :27-61 ). What August fails to teach is permitting the learner to 
select a desired learning program so that the first sentence is the first sentence of the selected 
learning program [Claims 2-4]. However, Jochumson teaches a system and method for handling 
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voice commands in interactive language learning functions, using speech recognition (2:51-63). 
Jochumson teaches a client browser displaying text, graphics, and multimedia data to a user of 
an interactive language learning system, and the user selecting a speech processing exercise 
via the displayed web page associated with the selected exercise (4:20-30). Jochumson further 
teaches where the user selects a speech recognition exercise on a displayed web page {a 
textual command} (4:43-53), or where the user selects a speech recognition exercise involving 
the user speaking, executing a speech capture thread {a voice command} (4:54-67). The 
system and method of August, which uses automated speech recognition to capture voice 
commands and textual commands to select from a list of words for practice would be used, as 
taught by Jochumson, to select a desired learning program. Therefore, it would have been 
obvious to one of ordinary skill in the art, at the time the invention was made, to have used the 
voice and textual commands of Jochumson to permit the learner to select a desired learning 
program, so that the first sentence is the first sentence of the selected learning program, in the 
interactive language learning system of August, in order to resolve user-spoken words or 
phrases into a command or control for an application, thus adding to the flexibility of using the 
system and method [Claims 2-4]. 

17. August teaches a system and method for interactive language learning, using both voice 
and textual commands to select from a list, as demonstrated above. What August further fails to 

4 

teach is wherein the voice command is converted into a textual command after the voice 
recognition engine has recognized the voice command [Claim 12]. However, Jochumson 
teaches where a plurality of language learning clients can send commands to a server, which in 
which a speech processing thread is responsible for accepting encoded audio, or speech, data 
packets from a client, decoding the audio packets into their original raw speech data, evaluating 
the raw speech data via a command and control speech engine, and transmitting a response, or 
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appropriate feedback, to the client, to be provided to the user (10:10-28), and where responsive 
feedback from the speech recognition program may be in the form of text (4:5-1 1). Jochumson 
further teaches where speech recognition is used to process voice commands (such as at 2:51- 
63). Jochumson thus teaches converting the voice commands of a user into text commands, a 
known technique that is applicable to the system and method of August. Thus, it would have 
been obvious to one of ordinary skill in the art to apply the technique of converting the voice 
command into text after being recognized, as taught by Jochumson, to improve the system and 
method of August, for the predictable result of displaying the voice command on the display of 
the interactive language-learning device [Claim 12]. 

18. Claims 6 & 13 are rejected under 35 U.S.C. 103(a) as being unpatentable over August, 
in view of Morse, III et al. (US 6,120,297), hereinafter known as Morse. 

19. August teaches all the features as demonstrated above in the rejection of claims 1 & 9, 
including output related to the pronunciation of a sentence (for any case including for words that 
have been mispronounced, the system can arrange the words into an order such as closest 
match through not well matched and as a default, can present the items needing most work at 
the top of the list. Word lists appear in the working window, 11:27-61), and a database holding 

■ 

words, sub-words, and groups related to a sentence (A word and punctuation list can be used to 
practice pronunciation skills, review definitions, etc. The list may be created from a lesson file, 
from a dictionary or other reference material, from the Internet, or from a list of sub-words, 
words, or groups pronounced inaccurately by the student, 1 1 :28-61 ). What August fails to 
explicitly teach is wherein the method further comprises the step of: the language learning 
system outputting full sentences, synonyms and phrases related to the pronunciation of the first 
sentence [Claim 6], and wherein the database further holds full sentences, phrases and 
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* 

synonyms that are closely related to each first sentence [Claim 13]. However, Morse teaches an 
interactive language learning device, having exercise involving selecting an appropriate 
synonym for a target word in a sentence (1 1:55-59), and an exercise involving selecting a word 
to complete a sentence, where the incomplete sentence is understood to be a phrase (12:32- 
38). The database and output comprised of words, sub-words, and groups of words, taught by 
August, would also contain full sentences, synonyms, and phrases, as taught by Morse, as all 
are merely words and groups of words. Therefore, it would have been obvious to one of 
ordinary skill in the art, at the time the invention was made, to have the language learning 
system of August outputting full sentences, synonyms and phrases related to the pronunciation 
of the first sentence, as taught by the exercises of Morse, and wherein the database of August 
further holds full sentences, phrases and synonyms that are closely related to each first 
sentence, as taught by the exercises of Morse, in order to provide background to the student for 
learning and retaining a target word's or sentence's proper definition and use [Claims 6 & 13]. 

Conclusion 

20. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. Kim (US 2003/0129571 A1) discloses a system and method for language education, 
which stores spoken sentences as text data. Raya et al. (US 2004/0241625 A1) discloses a 
method of teaching pronunciation via a telephone network, using a database of grammar files 
for various languages, using speech recognition to interpret voice commands to select relevant 
lessons based on language and level. Willetts (US 5,010,495) discloses an interactive language 
learning system allowing a student to select a text phrase, and recording his/her attempt at 
pronunciation of the phrase, and listen to a comparison of the recording with a digitized vocal 
version of the selected phrase. 
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Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Nikolai A. Gishnock whose telephone number is 571-272-1420. The 
examiner can normally be reached on M-F 8:30a-5p. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 



supervisor, Robert E. Pezzuto can be reached on 571-272-6996. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private 
PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you 
would like assistance from a USPTO Customer Service Representative or access to the 
automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 





NAG 

8/31/2007 



Rorerf E. Pezzuto 
Supervisory Patent Examiner 
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